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System and Method for Formulating, Delivering and Managing 
Data Concerning Programming Content and Portio ns Thereof 

This application claims priority under U.S. law to provisional U.S. patent 
applications serial no. 60/213,058, filed 6/21/00; serial no. 60/214,529, filed 6/27/00; 
and serial no. 60/231,180, filed 09/07/00, serial no. 09/751,702, filed 12/29/00, serial 
no. 09/751.809, filed 12/29/00, each of which applications are owned by the 
assignee of this application and each of which applications are incorporated in its 
entirety by this reference. 

Technical Field 

This invention generally relates to the field of programming control systems, 
and more particularly to systems and methods for allowing users to control and 
manage programming content. In one implementation, an application residing on a 
set-top box or other consumer electronic device, such as a television set, uses data 
synchronized to a program to dynamically block or modify portions of the program's 
video or audio based on a user's parental control settings. 

Background of Invention 
With the increasing availability of television programming, cable services, 
Internet programming, and other interactive entertainment services, there is an ever 
greater need for the ability to control and manage content of programming. Users 
desire to be able to more easily and effectively choose the types of television 
programs to which they and / or their children are exposed. 

In some existing "parental control systems," such as those associated with V- 
chip technology or Electronic Program Guides (EPGs), users may block all programs 
having particular ratings and content attributes. For instance, a user may block all 
programs having a rating of "TV-MA" (Mature) or all programs having the content 
attribute for "Violence.'' When a program with blocked ratings or content attributes 
so appears, both its video and audio are blocked for the entire duration of the program 
unless the user enters a personal identification number (PIN) to unblock it. With 
such existing systems, ratings and content attributes are generalized for the entire 
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program and rarely change. As a result, blocking is an "all or nothing" affair because 
either all of the program is blocked or none of it is blocked. This is a disadvantage if 
the ratings or content attributes apply to only a small part of the entire program. 
For instance, if a program contains a brief scene of nudity and the user 

5 blocked all programs with "Nudity," the entire program would be blocked even if the 
scene in question was only a few seconds in length. There would be no way to 
selectively block just that portion of the program. A good example of this is the 
movie "Titanic." After Titanic's release, a small firm began editing from Titanic a love 
scene between the two main characters. There was an enormous demand for this 

10 version of Titanic among parents and other users who did not desire to see that 
particular scene, but did wish to view the other portions of the movie. 

Even if parental control systems could block a portion of the program by 
briefly changing the program's ratings or content attributes or by changing the data 
being provided to the V-chip during a program, there would be no way to block just 

is the video or just the audio portion of the program. For instance, if a program 
contained a few seconds of adult language, the ratings and / or content attributes 
data would be unable to instruct the parental control system to block only the audio 
for that period of time. As a result, both video and audio would always be blocked 
together. 

20 Furthermore, existing parental control systems are also limited to completely 

disabling the audio or video. In other words, existing parental control systems 
cannot modify the video or audio in any way. For instance, if a program contains 
adult language, the ratings/content data cannot instruct the parental control system 
to switch to an alternative audio track containing a G-rated version of the dialog. 

25 A better approach would be to make the ratings and / or content data vary 

during the course of the, program. An even better approach would be to provide 
additional data to indicate whether the video, audio, or both video and audio portion 
of the program should be blocked. Additionally, the data could indicate ways in 
which to modify the video or audio portion of the program to make it acceptable for 

30 the current parental control settings. 
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Summary of Invention 
Certain terms used in this section are described more fully below, particularly 
in the "Terminology" session. The present invention addresses the problems 
discussed above by providing systems and methods for allowing users of programs 
5 to control and manage the ways in which portions of the programs are presented. In 
general, this invention provides program users with program data for a program 
corresponding to discrete portions of the program. The program data comprises (a) 
program content data, (b) program control data, or (c) both. The content data may 
consist of rating values associated with categories in a rating system and/or content 
10 attribute values indicating the existence of predefined attributes or traits associated 
with each discrete portion of the program. The program control data comprises 
information concerning the media component (e.g., audio, video, text, hyperlinks, 
etc.) to which the rating or content attribute applies. The program control data may 
also contain control instructions corresponding to the ways in which each portion or 
15 media component of the program may be disabled, modified, replaced, and / or 
edited. These data are delivered to a set top box or other consumer electronics 
("CE") device that couples to a display, like a television or monitor. 

Users can enter into the CE device a presentation profile, by which a user 
specifies content the user does not wish to review or wishes to insert over existing 
20 content. For instance, the user may select certain types of rating categories and / or 
content attributes that the user desires not to be presented. Optimally, users may 
specify a method for disabling, such as deleting, modifying, replacing, or editing, the 
portions of the program having the undesirable rating and / or content attribute. 
Based on the program data associated with the program and the information 
25 contained in the user's presentation profile, the CE device controls the programming 
presented for display. 

For example, the program data is compared against the presentation profile. 
When the presentation profile indicates deletion or editing of specific type material, 
and when the program data indicates the presence of that material in a portion of the 
30 programming, that portion of the programming is modified accordingly. If the 
program data contains a rating and / or content attribute listed in the presentation 
profile, systems and methods according to the present invention perform any.of the 
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following functions: block video, block audio, block video and audio, block only a 
portion of the audio or video, substitute local audio, substitute local video, substitute 
alternative audio stream, substitute alternative video channel / stream, substitute 
alternative video and audio channel / stream, edit content, and / or delete content. 

5 In one implementation, this invention uses streaming data or timestamped 

data synchronized to a program to provide ratings and / or content attribute 
information that vary during the program. This data is used by applications residing 
on a set-top box or other CE device to block or modify portions of a program's video 
or audio based on the settings selected by the user. In addition to providing varying 

10 ratings and / or content attribute information, the data may also include program 
control data related to what component of the program to block, such as video only, 
audio only, or both video and audio. Additionally, this data includes instructions on 
possible ways to modify the content. For example, the data could instruct the 
application to do such things as switch to an alternate audio track, switch to an 

is alternate video channel, or blackout a certain area of the screen. Alternatively, all of 
these different capabilities could be entered, enabled or disabled by the user via the 
presentation profile or a control. 

A further application of this invention is for Digital Video Recorders (DVRs) or 
digital disk recorders. When outfitted with this invention, DVRs or the like are 

20 capable of dynamically editing and deleting scenes based on the program data and 
the presentation profile created by a user. 

This invention aims to achieve one, combinations, or all of the following 
objectives: 

• To formulate program content data, associated with discrete portions of 
25 programming, which contains information related to programming content; 

• To formulate program control data, associated with discrete portions of 
programming, which contains information related to how programming 
may be edited, deleted, modified, substituted, or otherwise altered; 

• To provide, via a set-top box or other appropriate consumer electronic 
30 device, users with the content data, control data, or both; 

• To enable programming users to define those types of programming and 
portions of programming they desire not to be presented; and 
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. To enable users to change undesired types of programming and portions 
of programming by dialing, modifying, substituting, deleting, or editing such 
programming or portions thereof. 
Other objects, advantages and features of this invention will be apparent from review 
5 of the remainder of this document, including the drawings. 

Brief Description of Drawings 
Figure 1 is a diagram illustrating an embodiment of a system according to the 
present invention. 

io Figure 2 is a diagram illustrating one potential format of a programming signal 

and corresponding program data according to an embodiment of the present 
invention. 

Figure 3 is a diagram illustrating another potential format of a programming 
signal and corresponding program data according to an embodiment of the present 
is invention. 

Figure 4 illustrates an exemplary format for the data illustrated in Figures 2 
and 3 according to an embodiment of the present invention. 

Figure 5 is a diagram illustrating a presentation control system according to 

the present invention. 
20 Figure 6 is an exemplary menu screen of a program viewer interface 

according to the present invention. 

Figure 7 is an example of a presentation profile shown in Figure 7 according 

to the present invention. 

Figure 8 is a diagram illustrating an example of a program and corresponding 
25 program data according to the present invention. 

Detailed Description 

I. Terminology : 

Before further describing a particular implementation of the present invention 
30 that is shown in the drawings, the following terms are explained, although more 
thorough understanding of the terms can be reached by resorting to this entire 
document. These term explanations are not intended to be conclusive, as 
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technology will change and skilled persons will recognize other ways to implement 
the same functionality. 

• "Communications network" includes any network or infrastructure that 
supports communications between multiple devices, including broadband 

5 distribution networks, public or private packet-switched or other data 

networks, including the Internet, or circuit-switched networks such as the 
public switched telephone network and wireless networks. 

• "Consumer electronics device* or "CE device" is any electronic device or 
combination of devices capable of receiving, displaying, playing, 

10 presenting, recording, deleting, editing, substituting, or disabling audio and 

/ or video content Exemplary CE devices include: televisions, personal 
computers, cable set-top boxes, video cassette recorders, digital video 
recorders, digital video disc players, compact disc players, and personal 
computers. 

15 • "Content data" is any information corresponding to at least a portion of a 

program and related to the content of the program, including content 
ratings and content attributes, traits, or characteristics. 

• "Control data" is any information corresponding to at least a portion of a 
program and related to the manner in which the portion may be disabled, 

20 modified, replaced, deleted, edited, or otherwise modified by a CE device, 

set top box or the like. 

• "Presentation control system" is any device or combination of devices 
having (1) a receiver adapted to receive a program signal and (2) a 
processor for controlling the program to be displayed on a consumer 

25 electronic device, including a set-top box, television, or personal computer. 

• "Program 0 or "Programming" is any electronic presentation of information, 
such as text, audio, video, graphics, or any other form of multimedia, over 
a communications network. Exemplary programming includes: Internet 
programming, television broadcasts, cable television programming, pay- 

30 per-view programming, video-on-demand, interactive television 

programming, satellite programming, and multimedia presentations. 

• "Program data" means content or control data or other data associated 
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with or describing the content or manner of presentation of programming. 

• "Presentation preferences' refers to information related to the types of 
program content that either a person desires not to view or that a person 
desires to replace and/or the manner in which programs and portions of 

5 programs having this type of content may be disabled, modified, replaced, 

deleted, edited, or otherwise modified by a CE device. 

• "Service provider" is any entity that delivers programs to a program viewer, 
including cable providers, television broadcasters, satellite providers, and 
entities supporting Internetworld Wide Web ("Web") sites. 

10 II. System Overview 

Figure 1 shows components of a system 10 for implementing an embodiment 
of this invention. A transmission facility 12, such as a cable head end, is operated by 
a service provider and receives various types of programming from data / content 
providers 14, which may provide traditional broadcast or cable television 
is programming, video services such as pay-per-view, near video-on-demand, video- 
on-demand, promotional channels, or interactive television services. Exemplary data 
/ content providers 14 include BellSouth, (the assignee of this invention), Disney. 
Time-Warner, DirectTV, etc. In connection with the provision of some of these 
services, transmission facility 12 may also receive electronic program guide (EPG) 
20 data from data providers 1 6. A particular content provider 1 4, such as Disney, may 
also provide EPG data, thus acting as an EPG data provider 16. Also, EPG data 
provider 16 may include companies that specialize in EPG data, such as Gemstar, 
United Video, Starsight, etc. 

Transmission facility 12 processes the programming furnished by data / 
25 content providers 14 and 16 and provides to program viewers 18 a signal 1 3 over a 
communications network 15, which may be a broadband distribution network, a 
public or private packet-switched or other data networks (including the Internet), 
circuit switched networks such as the public switched telephone network, wireless 
networks, or any other desired communications network. Transmission facility 12 
may comprise a plurality of servers, such as broadcast server 19, video server 20, 
EPG data server 22, and program data server 24, for providing multiple types of 
programming and related data via signal 13. Program viewers 18 receive 
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programming at a presentation control system 26, which may be a standard set top 
box, a properly programmed processor, coupled to a display, a consumer electronic 
device, or any other device adapted to couple to communications network 15. 
Control system 26 includes, or couples to, a receiver (such as receiver 56 shown in 

5 Figure 5) for receiving and decoding signal 13 and a processor for controlling the 
program content to be displayed on devices such as television 28, personal 
computer 34 or any other suitable display device. 

Presentation control system 26 may be a stand alone system that 
communicates with a display device and other consumer electronic devices or may 

io be combined with the display device and / or any other suitable CE device in one 
device. In accordance with the present invention, presentation control system 26 
also may receive a presentation profile 35 from program viewers 18. Based on 
presentation profile 35 and program data 36 received from transmission facility 12, 
the presentation control system 26 manages and controls the presentation of 

is program content by interacting with video cassette recorder (VCR) 30, digital video 
recorder (DVR) 32, personal computer 34, or any other suitable CE device. 

As shown in Figure 2, signal 13 carries multiple channels 38, each occupying 
a different portion of a frequency spectrum of signal 13. Transmission facility 12 may 
provide program viewers 18 with different programs 40 throughout the day on each 

20 channel 38. Each program 40 may be divided into multiple consecutive data frames 
42, each of which contains all of the data related to the presentation of program 40 
for a particular time interval. 

In addition to providing program viewers 18 with program data 42 for each 
program 40, transmission facility 12 also provides program viewers 18 with program 

25 data 36 for each program 40. Data 36 is information associated with program data 
42 comprising program content data, program control data, or both. As shown in 
Figure 2, data 36 may be divided into multiple consecutive data frames 54 
corresponding to data frames 42 of program 40. Program content data may be 
information related to rating and / or content attributes, such as MPAA ratings, TV 

30 Parental Guidelines ratings, or any other rating system. Program control data may 
be information related to the manner in which program data 42 may be edited, 
substituted, deleted, or otherwise altered. For instance, in situations where program 

8 
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data 42 may be substituted, program control data may contain information about 
other channels or streams or other locations where replacement program data are 
present. 

As will be described in more detail below, program viewers 18 may use the 
information contained in data 36 about programs 40 to control and manage which 
programs are presented, which portions of programs are presented, and which 
media components are presented. Data 36 may also be used to control and manage 
the manner in which programs, portions of programs, and media components of 
programs and portions of programs are edited, substituted, deleted, or otherwise 
altered based on the information in data 36 and/or viewer preferences. 
III. Formulation and Delivery of Data : 

In accordance with the systems and methods of the present invention, data 36 
associated with program data 42 for programs 40 may be formulated by a number of 
entities. For instance, programming producers may formulate data 36 while 
producing programming. Alternatively, data 36 may be formulated by data / content 
providers 14, EPG data providers 16, by the entity supporting transmission facility 
12, or by any other entity that has access to programming. An example of a method 
of formulating alternative programming is one which uses the Internet Thus, a 
studio may release a portion or all of a story-line for a series on the Internet and 
allow users there to Vote" or suggest endings or twists in the plot line. Either before 
or after receiving such input, the studio may film the most popular of the suggestions 
and embed these alternative programs into other channels. If the alternatives are 
filmed first and then selected by viewers of the Internet sight via voting or similar 
feedback, the site can be linked to the transmission facility and, following tabulation 
of viewer feedback, the site can automatically instruct the transmission facility to load 
a particular alternative track. Ultimately, such viewer feed back may be done in real 

time or virtual real time. 

Program data 36 may be provided using many possible data delivery 
methods, as skilled persons will recognize. The following details several exemplary 

30 delivery methods: 

Timestamped Delivery: Data 36 may be provided using a timestamped data 
delivery approach. This approach delivers all data 36 for an entire program either 
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before the program begins or when the user tunes to the program. Timestamps are 
used to associate particular ratings and / or content attributes data to particular 
portions of the program. Ideally, this approach works best when presentation control 
system 26 has or couples to an internal clock that synchronizes to a network clock to 
5 assure accurate synchronization of data 36 to the program content. 

Batch EPG Delivery: Data used by existing EPGs is often delivered all at 
once, for example as a batch transmission. A similar approach can be used by 
which existing EPG data is supplemented with program data 36. Specifically, as 
shown in Figure 1 , an EPG data server 22 may receive data 36 from an EPG data 

io provider 16. Alternatively, the data provided by the EPG data server 22 may be 
enhanced by the addition of data provided by a program data server 24. The 
transmission facility 12 delivers the resulting data through the distribution network 15 
to presentation control system 26 or consumer electronic device 28, 20, 32, or 34. 
Streaming Delivery: Data 36 may also be provided using a streaming 

15 delivery method. This approach delivers program content data and program control 
data in real-time along with the program's video and audio content. There is no need 
for timestamps or clock synchronization since the data delivery is synchronized to 
the showing of the program. Methods of delivering streaming data include an MPEG 
private data stream or data insertion into the program's VBI (vertical blanking 

20 interval) line. 

An advantage of this approach over the timestamped approach described 
above is that it is not effected by delays in the video content. For example, if a 
program is delayed because a sporting event that precedes it goes into overtime, the 
program's timestamped data may no longer be accurate since it was downloaded 

25 ahead of time and assumed the program began at a certain time. Program viewer 
18 may tune to the program while in progress. 

Data 36 would have to be repeatedly retransmitted during the period for which 
it was valid because presentation control system 26 or consumer electronic device 
28, 30, 32, or 34 may not know what the last data values were when tuning to a 

30 program. Ideally, the retransmission interval should not be more than once a 
second. 

Regardless of the delivery vehicle, in operation, program viewers 18 receive 

10 
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signal 13 and data 36 from transmission facility 12. As described above, data 36 
may be included within signal 13 as another channel within the frequency spectrum 
or may be received as an independent data stream or as a periodic download. In 
situations where data 36 is received in the form of a periodic download before 
5 programs are viewed, data 36 may be stored in memory by system 26. In alternative 
embodiments, data 36 may be supplied to program viewers 18 as part of a broadcast 
television signal in the vertical blinking interval. Data 36 may also be supplied by 
other service providers using communications network 15 or any other suitable 
communications network. Data 36 may also be received via any public or private 
10 packet-switched or other data networks including the Internet, circuit switched 

networks such as the public switched telephone network, wireless networks, or any 
other desired communications infrastructure. 
IV. Data Formatting 

Data 36 provides program viewers 18 continuous information related to the 
is content of programs 42 throughout the entire presentation. As shown in Figure 4, 
data 36 may have a data structure comprising: 

(1) program content parameters, such as: timestamp 44, rating 46, content 
attribute 48, program component 50, and any other suitable parameter for 
identifying the content of program data 42 and 
20 (2) program control parameters, such as: control instruction or action 52 or 

any other suitable parameter associated with the manner in which program 
data 42 may be edited, substituted, deleted, or otherwise altered. 
Timestamp 44 associates program content and program control data with a 
particular time interval within a program 40. The time interval may be represented as 
25 an offset from the start of program 40 and may be expressed in hours, minutes, and 
seconds or in any smaller unit of time, such as the length of a video frame. 

The remaining parameters are used to define desirable control and 
management parameters for the corresponding time interval. For example, rating 46 
may be a value associated with a ratings system, such as the TV Parental 
so Guidelines ratings system, the MPAA ratings system, or any other standard ratings 
systems. Alternatively, rating 46 may be values associated with multiple ratings 
systems. Content attributes 48 may be a value associated with a trait or attribute of 
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program 40. Content attribute 48 may be based on the content rating system 
employed by the TV Parental Guidelines or MPAA rating system. Content attribute 
48 may be a value indicating the existence of certain attributes or traits in a particular 
time interval of program 40, such as violence (V), sexual situations (S), coarse or 
5 crude indecent language (L), suggestive dialogue (D) t fantasy violence (FV), or any 
other defined attribute or trait. A better approach than using the fairly simplistic 
content attributes associated with the existing TV ratings system (L, V, FV, S, D) 
would be to use a system that assigns numeric values to each of the content 
attributes depending on the degree of attributes present. For example, Violence (V) 

10 could be rated on a scale of 1-5 during the course of a program. Such ratings 
provide viewers with finer control over the level of violence blocked or modified. 
Program components 50 may be used to identify the media component of program 
rating 46 and / or content attribute 48 apply. For example, program component 50 
may indicate that rating 46 or content attribute 48 apply only to the video component, 

15 only the audio component, or both the video and audio components. Program 

component 50 may also be used where program 40 contains multiple audio and / or 
video streams within the same channel 38, in which case the data may indicate to 
which stream or streams rating 46 or content attribute 48 apply. Program control 
parameters, such as control instructions or actions 52, may be a value associated 

20 with the various ways in which program 40 may be blocked, disabled, modified, 

replaced, and / or edited. For instance, the following table I illustrates an example of 
possible control instructions and their corresponding functions supported by data 36. 
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Table I: Program Control Data 



uontroi vaiuc 


Control Function 


BV 


(jIUUix VIUCU 


BA 


DlUUft aUUlU 


BVA 




BP 


Klrkrlr nnh/ a nnrtirtn nf thp video diSDlaV 


SLA 




SLV 


substitute local video 


SAS 


substitute alternative audio stream 


SVC 


substitute alternative video channel / stream 


SAVC 


substitute alternative video and audio channel / stream 


DS 


delete scene 


SS 


substitute different scene 



As shown in Figure 2, data 36 may also be presented as a timestamped 
variance from a default setting. Data 36 may associate program 40 with a standard 

5 default setting, which is the same for all types of programs 40. Accordingly, only 
those portions of program 40 that vary from the default setting need to be assigned 
program content and control data. For example, the default setting may refer to 
programs that do not have any undesirable content and / or attributes. Using this 
method, data 36 need only contain information for those time intervals of program 40 

io that contain content and / or attributes that differ from the default setting and may 
reduce the amount of information in data 36. 

As shown in Figure 2, data 36 may also be delivered to program viewers 18 in 
real-time along with program 40. In this implementation, data 36 does not have to 
include information related to the time interval within program 40 because the two 

15 parallel streams are synchronized. Data 36 may comprise a plurality (e.g., more 
than two) of data frames 54, each corresponding to a discrete time interval of 
program 40. In this embodiment, data frames 54 may be synchronized with data 
frames 42 in program 40 such that the time interval associated with each is equal. 
Alternatively, the time interval associated with control data frames 54 may vary from 

20 that of data frames. 

V. Exemplary Data Format 

An example of a possible data format for dynamic ratings and content 
attributes is shown below in Table II. 
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Table II 


DEFAULT [rating1,rating2,.. 


. ] ;[, contentl, content2 t . . 


■] 


Timestamp [rating 1,rating2, . 


..],lcontent1,content2 f . 


. . ] [component [action] 


Timestamp [ratingl ,rating2, . 


..];[, contentl , content2, . 


..] [component] [action] ... 



In this example: DEFAULT indicates a set of overall ratings and content attributes 
5 for the program. These overall ratings and content attributes would be similar to the 
ones already provided by many existing EPGs and other parental control systems. 

• "Rating" is a value associated with a ratings system such as the TV rating 
system or MPAA rating system. However, other rating systems are 
possible and could be used. A program could have ratings from more than 

10 one rating system. Also, the program's rating value for a particular rating 

system could be missing or NULL since not all programs are rated. 

• "Content" is a value associated with a content attribute system (the word 
"traits" is sometimes used instead of "content") such as the ones 
associated with the TV rating system (V I S I L,D I FV) or MPAA rating system 

15 (e.g. Nudity, Violence, etc.). However, other content attribute schemes 

(such as the multi-level scheme proposed above) are possible and could 
be used. A program could have multiple content attributes from multiple 
content attribute systems. 

• "Timestamp" indicates when the associated ratings, content attributes, and 
20 actions begin. Ideally, the timestamp should be an offset from the start of 

the program. The timestamp could be expressed in hours, minutes and 
seconds (e.g. hh:mm:ss). A way should exist to subdivide seconds into a 
smaller unit of time such as frames (1/30 second). 

• "Component" refers to the part of the program to which the ratings and 

25 content attributes apply, such as audio, video, or both. "Component" may 

also refer to substitute channels, streams, or other locations of substitute 
audio and / or video. If a component value is not specified, both video and 
audio are assumed. 

• "Action" indicates ways in which the program could be modified to nullify 

30 the portion of the program with the given ratings and / or content attributes. 
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For instance, an action could instruct the presentation control system 26 to 
perform any of the functions described above in Table I. 

Table I ustrates how this data format could be used to deliver program data 

36 using the timestamped delivery approach described above. This example uses 
the existing MPAA ratings system (G.PG.PG-13.R.NG-17) and the content attributes 
associated with the TV ratings system (L = Language. V = Violence, S = sexual 
situations, D = suggestive dialog. FV = fantasy violence). 



10 



Table III: Example of Program Data 36 



DEFAULT R; L,V,S 



00:00:00:00 G 



00:11:15:02 R; L audio changeto 14 



00:11:16:11 G 



00:24:45:00 R; V,S 



00:24:45:30 PG-13; V video 



00:24:46:00 G 



00:32:01:00 R; S video mosaic 
20,100,30,150 



00:32:05:00 G 



15 



20 



Table ill shows where the content data has been formulated into a series of 
data records, each record associated with a particular portion of the programming. 
Thus, in this example: 

• "TV-MA;L,V,S" are the default ratings and content attributes for the entire 
program. These are similar to existing rating systems that use generalized 
ratings and content attributes for the entire program. 
. "00:00:00:00 G" is the first timestamp offset and first rating for the program. 
These values indicate that at the start of the program, the program contains 
material rated G and there are no content attributes present. If no 
ratings/content values are specified for the beginning of the program, the 
default ratings/content values should be used. 
. "00:1 1:15:02 R; L audio changeto 14" is the second timestamp offset. These 
values indicate that 11 minutes, 15 seconds, and 2 frames from the start of 
the program, the programs rating has changed to "R" and the show now has a 
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content attribute °L" which indicates coarse "Language". The "audio" value 
indicates that the rating is only associated with the audio portion of the 
program. "Changeto 14" indicates that an alternative audio track exists for 
that portion of the dialog. For the purposes of this example, assume that a 

5 character in the program has uttered a brief vulgarity. In this case, at the 

offset shown, presentation control system 36 may switch to an alternative 
audio track if the ratings and content attributes equal or exceed those 
indicated by the user's parental control settings. At the next timestamp, 
presentation control system 26 would reset the audio settings to those 

io indicated by the next ratings/content data. 

• "00:1 1 :16:1 1 G" indicates that the rating is back to G. Any blocking that 
resulted from the previous timestamp can be reset Using the example 
above, the character has stopped uttering the vulgarity and the audio can be 
returned to normal. 

is • "00:24:45:00 R; V,S" indicates that the rating has changed to R and there is 
now "Violence" and "Sexual Situations'. The lack of any component value or 
action value indicates that both video and audio should be blocked (disabled) 
if the ratings/content values equal or exceed the user's parental control 
settings. 

20 • "00:24:45:30 PG-1 3; V video" indicates that the rating has changed to PG-1 3 
and there is now only Violence (V). The Sexual Situations (S) have stopped. 
The "video" component value indicates that video should be blocked if the 
ratings/content values equal or exceed the user's parental control settings. 

• "00:24:46:00 G" indicates that the rating has returned to G. Turn off any 
25 blocking or modifications that resulted from the previous timestamp. 

• "00:32:01:00 R; S video mosaic 20,100,30,150" indicates that the rating is 
now R and there is a sexual situation (S). Let's assume there is some nudity 
on the screen. The component "video" indicates that only the video should be 
effected and the action "mosaic 20,100,30,150" indicates a mosaic effect may 

30 be used to blur the screen at the indicated coordinates. 

• "00:32:05:00 G" indicates that the rating is back to G. Turn off any blocking or 
modifications that resulted from the previous timestamp. 

16 
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In the above example, if the streaming delivery approach was used, the 
Timestamp" attribute would not be needed. As shown in Figure 3, program data 36 
would have to be retransmitted at some interval while the data is valid to 
accommodate users who tuned to a program in progress. Delivered data can also 

5 be buffered for retrieval. 

If program data 36 is batch delivered to presentation control system 26, an 
applet at the CE device or set top box associated with the user's display device may 
scan all of data 36 in advance to determine how often blocking would occur during 
the program. If implementing the user's presentation profile 35 based on the 
io received program data 36 would cause an unacceptably high amount of blocking or 
modification (e.g., in terms of amount of time program is blocked or percentage of 
program blocked), the entire program could be blocked or the user informed and 
allowed to change the rating. The user may select the threshold at which this 
decision is made as part of the user's input of presentation profile 35 data, 
is vi. Viewer Presentation Profile 

With reference to Figure 6, system 26 may include a viewer interface by which 
viewer 18 inputs a presentation profile 35 containing instructions for controlling 
programming to be displayed and the format thereof. The presentation profile 35 
that viewers may create allows more flexibility in using system 26 for controlling and 
20 managing programming. Viewer interface 58 and a template 59 allow viewers to 
enter data stored in a profile 35 that then operates with application program 60. The 
viewer interface 58 may be a standard remote control, a wired or wireless keyboard, 
control buttons on the CE device implementing system 26, infrared port, PDA port or 
other suitable communication device for communicating commands to the 
25 application program 60. By these viewer interface 58 devices or software, such as a 
viewer interface applet or the like, viewers can input information, such as information 
called for by template 59, into the CE device so that application 60 can populate a 
memory or other storage device with the users' presentation profile 35. Application 
program 60 enables program viewers 18 to interact with the template 59 and 
30 television 28 using a standard menu system to populate the presentation profile 35 

with the appropriate data. 

For instance, viewer 18 may configure system 26 permanently or temporarily 

| 
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to disable certain channels 38 in order to block undesired programming. In other 
words, application 60 running on a set top box or other CE device allows the user to 
turn parental controls on or off and indicate what ratings or content should trigger 
those controls. For example, if the user set the ratings level to R, all material with a 
5 rating of R or higher would be blocked or modified, as depicted in Table IV below. 



Table IV 



MPAA Rating 


Status 


X 


BLOCKED 


NC-17 


BLOCKED 


R 


BLOCKED 


PG-13 


Open 


PG 


Open 


G 


Open 



In addition to ratings, however, users may modify discrete programming 
10 portions having certain attributes. For example, if content attributes associated with 
the TV ratings system were used, viewers may indicate which of the following 
content attributes should be blocked or modified, as per Table V below in which the 
user blocks "Sexual Situations", "Suggestive Dialog" and "Language" content 
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Table V 



TV Content 


Status 


V (Violence) 




L (Language) 


BLOCKED 


S (Sexual Situations) 


BLOCKED 


D (Suggestive Dialog) 


BLOCKED 


FV (Fantasy Violence) 





Additionally, the system offers users the ability to block ratings or content attributes 
in several different ways. Specifically, the user could choose from the following 
5 blocking options for each rating level and content attribute: 

. Block the Entire Program . This would allow the user to use traditional 
parental blocking where the entire program is blocked if the overall 
(DEFAULT) rating or content attributes equal or exceed those set by the user. 
In this case, the dynamic ratings/content data would have no effect, only the 
10 DEFAULT ratings/content data would be used. 

. Block Portions . This would use dynamic ratings/content data to block video or 
audio that equals or exceeds the corresponding parental control settings. The 
DEFAULT ratings/content information would not be used, unless the dynamic 
ratings/content data was missing. With this option, modification of the 
1 5 program's content would not be allowed and any instructions to modify the 

program would be interpreted as an instruction to block the video or audio. 
• Block or Modify Portions . This would allow portions of the program's video or 
audio to be blocked or modified as indicated by the program's ratings/content 
data. The DEFAULT ratings/content information would not be used, unless 
20 the dynamic ratings/content data was missing. 

These options could be applied to individual ratings and content attributes. For 
example, the user could set blocking for content attributes as shown in Table VI: 
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Table VI 



TV Content 


Status 


V (Violence) 




L (Language) 


BLOCK OR MODIFY 
PORTIONS 


S (Sexual Situations) 


BLOCK ENTIRE PROGRAM 


D (Suggestive Dialog) 


BLOCK PORTIONS 


FV (Fantasy Violence) 





In this example, a program that had any sexual situations (as indicated by the overall 
ratings/content data) would be blocked in its entirety. Portions of the program with 
5 suggestive dialog (S) would be blocked, and portions of the program with language 
(L) would be blocked or modified. 

Table VII below shows a further example using a different, MPAA ratings 
system: 



io Table VII 



MPAA Rating 


Status 


X 


BLOCK ENTIRE PROGRAM 


NC-17 


BLOCK ENTIRE PROGRAM 


R 


BLOCK OR MODIFY PORTIONS 


PG-13 




PG 




G 





Even more detailed parental control settings are possible. For instance, viewer 18 
could choose to allow audio to be modified but not video, forcing video to always be 
blocked. Or, viewer 18 may select any of the following disabling methods: complete 
15 block, partial block, substitution from local device, substitution from service provider 
(alternate channel), and substitution from service provider (alternate stream), and 
local edit. 

In yet another aspect of this invention, viewers may add additional, 
demographic data to their presentation profile. The viewer may indicate the viewer's 
20 household income, number of children, ages, gender, etc. This demographic data, 
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as explained in applicant's concurrently filed application entitled, "Methods for 
Displaying Programming Content Based on User's Demographic Data, Usage Data 
and/or Commands," which application is hereby incorporated in its entirety by this 
reference, may be used by an applet running on the viewer's system 26 to select a 
particular program or advertisement targeted to viewers with a predetermined 
demographic profile. 
VI. Use of Program Data 

Figure 5 shows at least portions of a system 26 for controlling and managing 
the presentation of programming received via signal 13 based on data 36. The 
consumer electronics device that resides in a viewer's home and implements the 
system 26 can be provided with a blocking processor 66 adapted to communicate 
with a memory 62, a demultiplexer 64, and a control processor 68. Skilled persons 
will recognize that processors 66, 68 can be actual microprocessors or simply 
different portions of a single processor devoted to different tasks. Processor 68 (1) 
controls various components like tuner 64, blocking processor 66, signal processor 
70 and (2) communicates with one or more consumer electronic devices, such as 
VCR 30, DVR 32, personal computer 34, or any other suitable consumer electronics 
or home theater and / or entertainment device. An output signal processor 70 
outputs video and audio signals to television 28 or any other suitable display device. 
Skilled persons will recognize that these hardware components - which could be 
combined into one component - may be found in or added to traditional set-top 
boxes or consumer electronic devices and need merely be reprogrammed with the 
functionality described herein. 

In any event, in controlling programming content, system 26 acts on (1) 
program data 36 that has both content and control data, (2) presentation profile 35, 
or (3) a combination of the program data 36 and presentation profile 35. Examples 
of each of these are given below: 
A. Filtering Operations 

Data 36 is received at the control system 26. A processor, e.g., blocking 
processor 66, accesses profile 35 from a memory 62 and combines it with data 36 to 
determine if and how program 40 should be presented. As seen in Figures 7 and 8, 
for each time interval of program 40, blocking processor 66 compares data 36 



21 



WO 01/99416 



PCT/US01/17131 



against profile 35 and acts as needed. As described above in detail, data 36 
contains information for each time interval of program 40 related to various 
presentation control and management parameters, such as rating 46, content 
attribute 48, program component 50, controlinstruction 52. Profile 35 contains a list 
5 of rating / content attributes 74 that viewer 18 does not want to be presented and the 
corresponding disabling method 76. 

For example, referring to Figures 4 and 7, a blocking processor 66 compares 
rating 46 and content attribute 48 values in data 36 with the rating / content attributes 
74 in profile 35. If data 36 does not contain any rating / content attributes listed in 
10 profile 35, blocking processor 66 does not send any control instructions to control 
processor 68. Under these circumstances, control processor 68 then forwards 
channel 38 from demultiplexer 64 to output processor 70 where it may be presented 
on television 28. On the other hand, if data 36 contains a rating / content attribute 
listed in profile 35, blocking processor 66 then determines applicable instruction 76. 
Depending on the particular combination of program component 50 values in data 36 
and disabling method listed in profile 35, blocking processor 66 may send any of the 
control instructions 52 mentioned above and shown in Table I to control processor 
68. Implementation or exemplary instructions is described below: 

• Blocking processor 66 sends "block video" control instruction if viewer 1 8 
selects the "partial block" disabling method per instruction 76 and program 
component 50 is "video." 

• Blocking processor 66 sends a "block audio" control instruction if viewer 18 
selects the "partial block" disabling method per instruction 76 and program 
component 50 is "audio." 

• Blocking processor 66 sends a "block video and audio" control instruction 
if viewer 18 selects the "complete block" disabling method per instruction 
76 regardless of the program component 50. 

In response to any of these instructions from blocking processor 66, control 
processor 68 performs the appropriate signal processing to edit the signal in channel 
38 received from demultiplexer 64 and then forward the edited signal to output 
processor 70 for presentation on television 28. 
B. Editing Operations 
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In accordance this invention, programming may also be dynamically edited 
based on the viewer's presentation profile 35. For instance, blocking processor 66 
sends a "block portion of video" control instruction if viewer 18 selects the "partial 
block" disabling method 76 and program component 50 is "video." Thereafter, 
processor 68 may modify the signal in order to blank or blur the affected portion of 
the video display or may overlay the affected portion with any desirable text or other 
message. After completing signal processing, control processor 68 forwards the 
signal to output processor 70 for presentation on television 28. 

Alternatively, other programming content, ranging from audio, to video, to 
Web pages or combinations thereof, may be substituted for the undesired 
programming portion. For instance, blocking processor 66 sends a "substitute local 
audio" control instruction if viewer 18 or the transmission facility 12 has selected the 
"substitution from local device" disabling method per instruction 76 and program 
component 50 is "audio." In response to this instruction from blocking processor 66, 
control processor 68 performs the appropriate signal processing to block the audio 
component of channel 38 received from demultiplexer 64. Control processor 68 
then initiates a communications link with (a) a different channel having the substitute 
audio or (b) the desired local device, such as memory 62, DVR 32, VCR 30, 
personal computer 34, or any other suitable device capable of providing audio, from 
which the substitute audio is to be received. After receiving the substitute audio, 
control processor 68 performs the appropriate signal processing to combine the 
substitute audio with channel 38 and then forwards the combined signal to output 
processor 70 for presentation on television 28. 

Other commands such as substitute video, substitute local video, substitute 
video from memory 62, switch channel, display Web page, etc. may be implemented 
by the processors 66, 68 in similar fashion. Note that alternate content may be 
provided from transmission facility 12 as part of the original signal 13, either in the 
form of alternate channels 38 or alternate streams within a channel 38. Program 40 
on channel 38 may comprise a plurality of audio streams and a plurality of video 
streams, as well as a plurality of alternate audio streams and a plurality of alternate 
video streams. In this way, there may be an alternate stream according to each 
rating / content attribute category. In these situations, the transmission facility 12 
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includes within the data 36 information concerning the channel or URL to which the 
system 26 should tune for the alternate content. Alternatively, the alternate content 
may also be provided by a service provider other than the service provider supplying 
signal 13. 

5 Referring to Figures 7 and 8, the systems and methods of the present 

invention will now be described with reference to a particular example. Suppose a 
particular viewer desires to view a program 40 containing the movie Titanic. As 
shown in Figure 7, suppose the viewer has configured a profile 35 to completely 
block any programming content or portions of programming content containing nudity 

o or associated with a rating of R or X and to perform a local edit for portions of 

programming content rated PG-13 and containing sexual situations. Also, as shown 
in Figure 8, suppose the movie Titanic has been divided into multiple consecutive 
data frames 42 for each discrete time interval for the duration of the movie. 

Recall from above, the movie Titanic would have been rated PG except for 

5 the short love scene between the two main characters. Thus, prior to the present 
invention, a viewer that desired not to view programming rated PG-1 3 would not 
have been able to view any portion of Titanic. The present invention solves this 
problem. For example, in the present invention data 36 associated with each portion 
of Titanic is also provided to the viewer. Data 36 comprises rating data, content 

o attribute data, and control instructions. As shown in Figure 8, each data frame 

associated with Titanic was rated PG and had a general content rating except for the 
data frame associated with the love scene. This data frame was rated PG-13 
because of the sexual situation. In this circumstance, where profile 35 was 
configured to perform a local edit on content rated PG-13 and containing sexual 

5 situations, the systems and methods of the present invention enable the program 
viewer to view each portion of the movie except the portion rated PG-13 and 
containing the sexual situation. In addition, because profile 35 was configured for 
"local edit", the systems and methods of the present invention enable the program 
viewer to edit out the undesirable portion using DVR 32. 

o VII. Other Features and Embodiments : 

Other features may be programmed into the control system 26 that may 
reside on the CE device or set top box coupled to a display device. For instance, it 
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may not be immediately apparent to viewers whether the control system 26 is in 
effect when they initiate the display device and tune to a program. On screen 
indicators may be used to indicate the status of the controls (e.g., an icon may 
display or a message like "All programming with ratings above PG-13 have been 

5 blocked."). Likewise, during modification of the programming content, an icon may 
indicate to the user that the content has been edited or deleted. Further, in 
situations where viewers have selected the alternate programming to be displayed, 
e.g., as part of voting with other viewers via previewing of the programming on the 
Internet or the like, icons or other indications can be displayed to the viewers 

[0 indicating that they selected the alternative scene or otherwise identifying the source 
of the alternative scene. 

PIN entry or other security protocols can be implemented by system 26 to 
allow valid users to initiate, modify or disable the controls. Correct PIN entry shows 
the programming in its unmodified form. Programming controls can be reinstated 

is when the user tunes to another channel or turns off the Set Top Box or CE device. 
While this invention has been described in conjunction with specific 
embodiments, it is evident that many alternatives, modifications, and variations will 
be apparent to those skilled in the art. Accordingly, the preferred embodiments of 
the present invention described are intended to be illustrative and not limiting. 

20 Various changes may be made without departing from the true spirit and full scope of 
the invention as defined in the following claims. 
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What is claimed is: 

1 1 . A system for controlling and managing presentation to viewers of television, 

2 cable, satellite, Internet, broadcast or other programming content the system 

3 comprising: 



4 a. a receiver adapted to receive a signal corresponding to a program and 

5 data associated with at least a first portion of the program, the data 

6 comprising (1) content data, (2) control data, or (3) both; 

7 b. a viewer interface adapted to receive information related to program 

8 presentation preferences of a viewer; and 

9 c. a processor adapted (1 ) to modify the content of at least the first 

0 portion of the program based on the data and program presentation 

1 preferences and (2) to output the modified first portion for presentation 

2 on a display device with the remainder of the program. 



1 2. The system of claim 1 wherein the processor is further adapted to modify the 

2 first portion by blocking an audio, video or text component of the program. 

1 3. The system of claim 1 wherein the processor is further adapted to delete the 

2 first portion. 

1 4. The system of claim 1 wherein the processor is further adapted to substitute 

2 alternate programming for at least the first portion. 

1 5. The system of claim 4 wherein the signal carries the alternate programming 

2 via a channel comprising programming having audio, video or both audio and video 

3 components. 

1 6. The system of claim 1 wherein the viewer interface comprises an applet 

2 adapted to run on a consumer electronic device coupled to or incorporating the 

3 processor, wherein the applet causes the display device to show the viewer interface 
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7. The system of claim 1 wherein the viewer interface comprises a wireless 
remote control device that communicates with the processor and a memory 
associated with the processor. 

8. The system of claim 7 wherein the processor is adapted to replace the first 
portion with alternate programming. 



, 9. The system of claim 8 further. comprising an external device, adapted to 
2 communicate with the processor, the external device selected from the group 

consisting of a memory, a DVR, a VCR, an Internet browser, a tuner for tuning within 
the signal to a second channel having an alternative audio or video component 

10. The system of claim 1 further comprising a transmission facility for formatting 
the programming, wherein the transmission facility comprises: a program server for 
providing programming content and a data server for providing 1) content. 2) control 
data or 3) both content and control data associated with multiple discrete portions of 
the programming content. 



! 11. A method for controlling and managing presentation of programs and portions 

2 thereof, comprising: 

3 a. receiving a program; 

4 b. receiving data associated with the program, wherein the data 

5 comprises (1) content data, (2) control data, or (3) both content and 

6 control data; 

7 C. 



receiving information related to the program presentation preferences 

8 of a program viewer; ■ 

9 d . modifying the content of at least a portion of the program based on 1) 

10 

n 

12 e . displaying the modified content. 



IMVViii^Hig *. — 

the data associated with the program and 2) the program presentation 
preferences of the program viewer; and 
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1 12. The method of claim 11 wherein modifying the content comprises blocking the 

2 portion of the program. 

1 13. The method of claim 12 wherein modifying the content comprises blocking 

2 either an audio or video component of the portion of the program. 

1 14. The method of claim 11 further comprising replacing (1) the at least a portion 

2 of the program and (2) a second portion of the program with alternate programming. 

1 15. The method of claim 14 wherein the alternate programming comprises an 

2 alternate audio or video track contained on a separate channel. 

1 16. The method of claim 1 1 further comprising displaying indicia to viewers to 

2 indicate (a) whether the portion of the programming will be modified or (b) each 

3 portion of the programming that has been modified. 

1 17. The method of claim 1 1 wherein, before modifying the content, the method 

2 determines whether the portion to be modified exceeds a selected threshold 

3 percentage of the program. 

1 18. A method of formulating programming and data describing the programming 

2 for delivery to viewers over a communications network, the method comprising: 

3 a. receiving a program; 

4 b. associating with each of multiple, discrete segments of the program a 

5 data record comprising (1) content data, (2) control data, or (3) both 

6 content and control data; and 

7 c. delivering the program and the data record either (1) to a transmission 
s facility for forwarding to viewers over the communications network or 

9 (2) directly to a consumer electric device associated with a display for 

10 showing programming to the viewer. 

l 1 9. The method of claim 1 8 wherein the content data comprises information 
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related to (1) a rating category, (2) a content attribute, or (3) both a rating category 
and content attribute. 

20. The method of claim 1 9 wherein the content data further comprises 
information related to a predefined level of the content attribute. 
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21 The method of claim 1 8 wherein the control data comprises information 
indicating whether the content data applies to an audio, video, or text component of 
each segment of the portions of the program. 



1 22. The method of claim 18 wherein the control data comprises control signals by 

2 which at least one component of one segment may be modified based on the 

3 content data. 



l 23. 



The method of claim 22 wherein the control signals are selected from the 



group consisting of (1) block video, (2) block audio, (3) block video and audio, (4) 
block only an area of the video, (5) substitute local audio, (6) substitute local video, 
(7) substitute alternative audio stream, (8) substitute from alternative video channel, 
(9) substitute alternative video and audio channel, (10) delete, and (1 1) substitute 
6 alternative programming from external device. 

1 24. A method for providing dynamic user control over programming delivered via 

2 a cable, broadcast, telecommunications, packet-switched or other network to a 
user's home that is equipped with a consumer electronics device for at least 
displaying the programming content, the method comprising: 

a) providing the consumer electronic device with data synchronized to the 
programming, wherein the data at least comprises varying content data; 

b) inputting the user's control settings that describe the type of programming 
the user desires to modify; and 

c) modifying the programming according to the user's control settings and 
l0 displaying the modified programming to the user. 
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1 25. The method of claim 24 in which the modifying step is performed by using a 

2 DVR device dynamically to store and edit the programming in order to seamlessly 

3 display the modified programming to the user. 



1 26. The method of claim 24 in which the content data is delivered to the user's 

2 consumer electronic device via the signal that carries Electronic Program Guide 

3 data. 

1 27. The method of claim 24 further comprising the step of determining whether to 

2 display a particular program by comparing all content data associated with the 

3 particular program to the user's control settings and a preset threshold. 



1 28. The method of claim 27 in which the comparing step further comprises the 

2 step of determining the amount of the particular program to be blocked and 

3 comparing that amount to a preset threshold. 

1 29. A method for formulating and delivering data for use in controlling the 

2 presentation of a program and portions thereof, the method comprising: 

3 a. formulating frames of program data and associating each frame with a 

4 discrete portion of the program, wherein the program data comprises 

5 (1) content data, (2) control data, or (3) both content and control data; 

6 b. delivering the program data to a transmission facility; 

7 c. distributing the program data to viewers. 
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I 



30. The method of claim 29 wherein the content data describes attributes of the 
discrete portion of program to which the content data is associated. 



l 31 . The method of claim 30 further comprising the step of distributing alternate 
scenes for use in the program. 



32. The method of claim 31 in which the alternate scenes comprise an alternate 



2 audio track and alternate video track. 
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33. The method of claim 32 further comprising replacing a first discrete portion of 
the program with alternate programming when the attribute associated with the first 
discrete potion exceeds a default threshold. 



i 34. The method of claim 33 in which the alternate scenes are selected by 



viewers. 



35. The method of claim 34 further comprising displaying indicia to viewers to 
indicate when a discrete portion of the program has been modified to display an 
alternate scene. 



! 36. The method of claim 35 further comprising displaying to a selected viewer an 
2 indication that the alternate scene was chosen by the selected viewer. 



37 The method of claim 29 in which the distribution of the program data occurs 
by embedding the program data in the channel carrying an Electronic Programming 



Guide. 



l 38. 

2 
3 

1 



. u . The method of claim 29 in which the program includes at least one targeted 
advertisement and the content data includes demographic data identifying the 
demographics of the viewers to whom the targeted advertisement is directed. 

39. The method of claim 38 further comprising the steps of collecting presentation 

> preferences from at least one viewer, comparing the program data with the 

3 presentation preferences and presenting the program in accordance with the results 

4 of the comparison step. 

, 40 The method of claim 39 in which the presenting step is accomplished by 

0 locally editing the program to replace a discrete portion thereof with the targeted 
3 advertisement appropriate to the demographics of the at least one viewer. 

1 41 . A method of remotely controlling presentation of a program to a viewer, the 
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